Most motion pictures produced for conventional theatrical exhibition are photographed at the frame rate of twenty-four frames-per-second, which has been in use for most of the history of cinema. At that frame rate, there persist undesirable artifacts that detract from the presentation of high-quality images, which is one of the goals of modern motion picture production and presentation. Movies shot and presented on film at 24 frames-per-second often have flicker, perceptible grain and motion artifacts, and an appearance that lacks immersive character. This latter artifact is particularly objectionable on large theater screens (fifty feet or more in width), since any image component appears to move a greater physical distance between discrete images on a large screen than on a smaller screen.
Films shot at higher frame rates succeeded, to some extent, in suppressing these undesirable artifacts. Cinerama (three synchronized 35 mm, 6-perforation filmstrips, photographed and projected at twenty-six frames-per-second, primarily used for travelogues) was somewhat successful in reducing flicker and improving resolution, and was immersive. The original Todd-AO system utilized photography at thirty frames-per-second, along with the conventional twenty-four, shot with another camera. A few motion pictures, including Oklahoma and Around the World in Eighty Days, were produced in that format. However, the use of separate photography at two different frame rates was costly, and few films were produced in the Todd-AO format. More recently, the inventor herein taught a means for transitioning between film sequences photographed for exhibition at twenty-four frames-per-second and other sequences photographed at thirty frames-per-second between scenes of a single motion picture (U.S. Pat. No. 5,096,286 (1992)). While there was a perceptible improvement in smoothness of motion at the higher frame rate, the latter system was never developed commercially.
More recently, Weisgerber has taught the use of forty-eight frames-per-second as a rate for photographing and showing motion pictures, to develop a presentation that suppresses the undesirable artifacts present at twenty-four frames-per-second (U.S. Pat. No. 5,627,614 (1997)). In that invention, certain sequences or certain image components are photographed at forty-eight frames-per-second, for a “high-impact” presentation. Other sequences or image components are photographed at twenty-four frames-per-second and double-frame printed, to retain the artifacts that gave the film the “legacy” look. With the entire motion picture produced according to that invention projected at forty-eight frames-per-second, it became possible to give certain portions of a motion picture film or certain image components a more realistic look, compared to other portions of the film or other image components.
The method of Weisgerber 614 works well only for new motion pictures, produced according to it. Because it calls for certain scenes or sequences to be photographed at 24 frames-per-second, and others to be photographed at 48 frames-per-second, it cannot be used to enhance motion pictures that were produced previously at the traditional rate of 24 frames-per-second. Weisgerber 614 called for certain portions of a motion pictures to be photographed at that speed and double-frame printed, but if a previously-existing motion picture photographed entirely at that frame rate were double-frame printed, it would still have the appearance of motion captured and displayed at 24 frames-per -second. Therefore, the method taught in Weisgerber 614 cannot be utilized for upgrading previously -produced motion pictures.
The theatrical cinema community is now recognizing and embracing the advantages of high-frame -rate presentation. The literature has recently featured high-frame-rate (HFR) photography and presentation as a topic of interest. Examples include: HFR Reactions, a SMPTE News Brief from Jan. 4, 2013, Goldman: Higher Frame Rates for 3-D: SMPTE Newswatch, November, 2012 (Peter Jackson's latest The Hobbit release and Douglas Trumbull's conversion of his ShowScan system to digital; “Playing HFR content in . . . theaters will therefore be possible, but not quick, simple, uniform or seamless”), Kaufman: The Aesthetics of High-Frame-Rate Cinema, Creative Cow Magazine (2012, quoting Steven Poster, President of the International Cinema Guild: “with the benefit of accumulated experience, the majority of moviegoers will adjust to the new language of 60 fps, and higher, motion picture, and will look back on the only 24 fps world in the way that we look back on black & white movies at 18 fps; quaint artifacts of another world.”), “The Future of Cinema is Smooth, Crisp and Amazing,” a slide show for exhibitors from www.christiedigital.com, March, 2012: “The biggest blockbusters are usually about immersive experiences and escapism—big, vibrant, high-action motion pictures” (at 2), “Filmmakers are now demanding the motion picture industry adopt HFR standards that reduce the motion blur, judder and strobing that's now common, and instead deliver enhanced/immersive 2D and 3D experiences for movie-goers. HFRs will give the tools and delivery platform they need to fully realize creative ideas” (at 4), “Digital cinema has turned into a fast-moving train, impossible to stop and largely driven by the box office performance of 3D movies” (at 6).
In an interview on May 12, 2012 for in70 mm.com, high-frame-rate pioneer Douglas Trumbull (inventor of the ShowScan 60-fps system, U.S. Pat. No. 4,477,160 (1984)), interviewer Wolfram Hannemann mentioned Trumbull's picture Silent Running (1972) and its production in the standard aspect ratio for 35 mm film, and asked: “So was it due to budgetary reasons that it was only 1:1.85?” Trumbull replied: “Yes—that was not enough money to make a big spectacle, in 70 mm.” Trumbull also posted a statement on his web site, www.douglasstrumbull.com, on Jan. 11, 2012. Referring to James Cameron's digital production of Avatar, he said: “It does reveal . . . a frightening lack of industry-wide understanding of how to move forward toward a more powerful cinematic form and format that is more immersive.”
Jeffrey Wells wrote in Hollywood Elsewhere on Apr. 29, 2012 (found at www.hollywood-elsewhere.com): “The bottom line is that, once younger audiences get a taste of it, 48 frames-per-second will be here to stay.” This quotation sums up the call in the industry for higher frame rates, particularly 48 frames-per-second. All of these quotes were from articles written during the past year, which demonstrates that the motion picture industry wants high-impact presentation, with the quality associated with the 70 mm film format, at high frame rates, and produced at an affordable cost. As shall be shown, the present invention meets all of those needs.
If the predictions in the literature are accurate, a highly immersive presentation at a high frame rate will lure young viewers, who comprise the majority of today's audiences, into motion picture theaters. Only the market segment that enjoys the “classics” or motion pictures which utilize the legacy look for creative purposes would accept conventional 24-frame-per-second exhibition of films that were originally photographed at that frame rate. Emerging moviegoers will prefer the experience of watching motion pictures shot at a higher frame rate, rather than produced at the conventional frame rate of 24 frames-per-second. All the other aspects of “legacy” films, including the cinematography, story lines and talent of the performers, will be less desirable to future audiences, unless those motion pictures can be upgraded to provide the sort of immersive presentation that those audiences will expect.
The predictions in the literature indicate that only a motion picture that delivers the appearance of an immersive, high-frame-rate presentation will give contemporary audiences what they want. The upgrade described here would prevent the theatrical distribution value of essentially all of the motion pictures produced until now from being diminished by changing audience tastes, despite whatever artistic or other cinematic merits those motion pictures might have. Therefore, it the primary object of this invention to enhance motion pictures photographed and exhibited at a historic frame rate of 24 frames-per-second, and give them the exact appearance that they would have had, if they had originally been photographed and exhibited at the higher frame rate of 48 frames-per-second.
With the recent changes in motion picture technology, along with changes in the desire of contemporary viewers of motion pictures to experience a more immersive presentation than is available at 24 frames-per-second, it has become essentially impossible for previously-produced motion pictures to give these viewers the experience they are beginning to demand. This trend will become more apparent as more motion pictures are produced and exhibited at 48 frames per second or other high frame rates. In effect, the time-honored rate of 24 frames-per-second may soon become a part of cinema history. It will represent a “legacy” look that will appeal to the market segment that like “classic films” and enjoy that specific type of experience, but it will not appeal to the larger market segment that will expect a more immersive motion picture. These viewers will expect the clarity of images and the smoothness of motion in action scenes that is only available with higher frame rates.
The recent changes in motion picture technology will create a sharp line of demarcation between any motion picture produced in the future at a high frame rate, and any motion picture produced at the traditional frame rate, even if such movies were made recently. Without a means for previously-produced motion pictures to deliver the sort of experience available with new motion pictures, those movies will be regarded as part of cinema's past. Therefore, it is the primary object of this invention to provide a means for upgrading previously-produced motion pictures to deliver the quality of experience that is becoming available with new motion pictures.
The invention also does this in an economical manner, which is also one of its objects. If a motion picture was photographed at 24 frames-per-second, and a producer later decides to upgrade it for exhibition at 48 frames-per-second for any reason, the method disclosed here accomplishes that upgrade. In addition to its ability to upgrade previously-produced motion pictures, the invention can also be used to upgrade sequences of stock footage, on film or digitally-captured, for use with motion pictures specifically made for exhibition at high frame rates such as 48 frames-per-second. It can also be used for animation produced at 24 frames-per-second, as well as live action, resulting in a dimensionalized, multi-plane experience. The method eliminates judder and other artifacts of 24-frame-per-second production, including with CGI animation at that frame rate.
Most previously-existing motion pictures, even if they were produced recently, do not even meet the 1990s standard of Weisgerber 614. Most commercially produced films were, and still are, photographed at twenty-four frames-per-second, in the 35 mm feature film format, with four perforations per frame. That format did not allow for sufficient visual information storage to deliver the full effect of the previous Weisgerber invention, which required the 70 mm film format.
Under the present invention, films produced in the 35 mm format can be converted to a digital platform that can exceed the 70 mm film standard for this purpose, with exhibition by digital projection, although film projection could be used. Most importantly, twenty-four discrete images displayed during each second are not enough to provide for the smooth appearance of motion which the previous Weisgerber invention strove for, but could not fully deliver, since additional images would have been required. Unfortunately, analog film methods could not have captured images that would have displayed the look that a motion picture would have had, if it had actually been photographed at a higher frame rate. Film production and exhibition also adds undesirable amounts of judder and blurring to the transitional “in between” images that are blended through conventional image compositing. For this reason, the present invention does not blend pre-existing images at any time, but renders entirely new images for interpolation between original images, thereby doubling the amount of picture information contained in a motion picture.
The 48 frame-per-second frame rate is essential to the practice of the invention, and it can be used to upgrade any motion picture photographed at 24 frames-per-second, no matter when it was produced, or whether it was originally photographed on film or by digital image capture. With the upgrade to 48 frames-per-second for release, a motion picture enhanced according to the invention described here can also possess enhanced commercial value, because it will be distributed more widely, resulting in an additional revenue platform.
The invention disclosed here is resolution-independent, in that it can be used in connection with any resolution level presented to the viewers of a motion picture, including newly-developed high-resolution formats.
Smoothness of motion, as observed by the audience, is especially important in modern, large-screen motion picture theaters. These screens are, in many cases, fifty feet or more in width. Any finely-delineated motion projected onto such a wide screen must appear to traverse a greater distance on that screen between images than is traversed on older screens, which were seldom over forty feet wide. In order to display motion at 48 frames-per-second so it appear smooth, it is necessary to add a new extra image between each of the original images of a motion picture. This way, the large distance displaced by each picture element (pixel) from one image to the next does not impart a jerky appearance to the motion picture as projected. The present invention solves this problem by replicating exactly what each new, rendered image would have depicted if each of those images had been original images in a motion picture photographed at a high frame rate such as 48 frames-per-second.
It should be noted that other inventions have used variable frame rates for storage of digital motion picture images; see, e.g. Cok, U.S. Pat. No. 7,242,850, which teaches a method for storing such images at different frame rates for economy of storage space. The present invention teaches away from Cok by adding data in the form of new digital images, each such image placed between each image pair from the original motion picture. It is true that Cok manipulates frame rates by digital means in the practice of his invention, but his method works against the method disclosed here. Cok is drawn specifically to storing digital images with maximum efficiency and conservation of storage space. By doing so, he sacrifices the image texture, image quality and resolution that the present invention delivers. For example, Cok would only require one exposure of a shot to establish a location, even though that shot would be displayed on the screen for several seconds. When displayed, Cok's single exposure would exhibit a freeze-frame appearance, which lacks the texture and quality that would be present if the same shot were exhibited in multiple exposures, photographed at a frame rate that is the same as the display rate. Cok also calls for the blending of images to create the new images that he interpolates into the digital image sequences which he processes. It is an important feature of the present invention that there is no blending of images, and that new images are rendered for interpolation only by estimating the motion of each pixel and producing images that place each pixel where it would have been, if it had been captured at a moment in time exactly half-way between the instant at which the previous image in the original motion picture and next image in the original motion picture were captured.
By the Cok standard, the invention disclosed here is wasteful. The object of the present invention is to enhance image quality and resolution for viewers of motion pictures, primarily by taking previously-produced motion pictures produced at 24 frames-per-second, and upgrading them for exhibition at 48 frames-per-second. The invention is based on rendering a new image and inserting it between each image of the original motion picture and the next image in sequence. By doing this, the method disclosed doubles the amount of picture information contained in the motion picture. This is an inherent feature of the invention. There is a tradeoff between economical storage of data and increasing the amount of picture information to provide an enhanced viewing experience for audiences in motion picture theaters. The two goals are incompatible and militate in opposite directions. Cok optimizes efficient data storage, while the invention described here optimizes the viewer's experience by doubling the amount of information in a motion picture. By the Cok standard, the invention described here would be considered extremely wasteful. In effect, Cok compresses image data, while the present invention expands it.
The present invention also constitutes a significant improvement over Cok, because it takes advantage of technology that was unknown when the Cok invention was filed in 2001. Since that time, image quality has improved to the point where it is now possible to accomplish lossless image conversions. Digital files can be transferred and converted with no loss of resolution or image quality. Because of newly-available levels of efficiency in the processing of digital images, these transfers can be considered “lossless” in that new frames that are renderd by digital means look as good, in every aspect, as original frames from a previously-produced motion picture. The present invention teaches a method for rendering new frames and interpolating them between each frame of the original motion picture and its successor. For such an invention to work properly, the new frames must look as good, in every aspect, as the original frames. New efficiencies of digital data transfer allow this, but it could not have been contemplated when Cok reduced his invention to practice in 2001.
Even if the method taught by Cok were used for the purpose of the present invention, it would not succeed in delivering an appropriate cinematic presentation, free of artifacts. The mere use of variable frame rates, as utilized for slow-motion effects, is unsuitable for a high-impact presentation. One of the primary deficiencies of the prior art is that existing variable frame-rate methods are not capable of dealing effectively with complex motion, so that such motion could be presented to an audience in a form that is free of undesirable artifacts. By using software that “looks ahead” to place each pixel of each image in its proper position, the invention goes beyond the prior art and overcomes this deficiency.
Prior methods of synthesizing digital images also fail to produce the quality of images required for the practice of the present invention. For example, the method taught by Hazra, U.S. Pat. No. 6,369,614 and assigned to Intel Corp., may fulfill its object for images to be shown on a computer screen. On large motion picture theater screens, however, it fails to provide the resolution, smoothness of motion or image clarity, which are the objects of the present invention. For the smooth motion that contemporary audiences are accustomed to experiencing, due to its presence in other media, each pixel of a motion picture must appear to move only a short distance on the screen from one image to the next. On a screen fifty feet (600 inches or 1524 cm) wide with an image 2080 pixels wide, each pixel is nearly three tenths of an inch or three quarters of a centimeter wide. On such a large screen, it is critical that the apparent distance traveled on the screen between images be minimized, in order to display motion that appears smooth and free of unwanted artifacts. A rate of 48 frames-per-second allows each pixel to appear to move only half as far between images as a rate of 24 frames-per-second.
The primary difficulty with the method disclosed in Hazra 614 is that, if an image is found to be of unacceptable quality, Hazra's method calls for “aborting and repeating a previous frame” (claims 5, 17 and 21; see also claim 7). The feature that repeats entire frames when the quality of a particular frame is deemed unacceptable defeats the entire purpose of the present invention. If enough frames are repeated in a sequence, the result displayed to an audience would be similar to double-frame printing, as used in Weisgerber 614 for scenes or sequences of a motion picture when a high-impact audience effect is not desired. In other words, the Hazra method cannot prevent the result that portions of motion pictures are absolutely unacceptable for the present invention. The present invention corrects that defect by processing most of the pixels in the frames which the Hazra method would consider “unacceptable” and repeat in their entirety. In that manner, as shall be shown, the present invention delivers the desired effect, in a way that the Hazra invention cannot. This correction feature, which is an integral part of the invention described here, could not have been anticipated by the prior art.
The present invention presents the appearance of immersive, high-impact motion pictures by enhancing previously-produced motion pictures to render the image quality typically associated with advanced motion picture technology. By using contemporary computer techniques, motion pictures that were photographed at twenty-four frames-per-second can be enhanced to appear as though the motion had, instead, been captured at forty-eight frames-per-second. This allows release of previously-produced motion pictures, with an image quality that will satisfy contemporary audiences. Thus, the motion pictures treated according to this invention can generate a new revenue stream for their owners. The enhancement of motion pictures as described here is demonstrated by a working prototype of the system used in the practice of the invention, installed in a commercial motion picture theater.